perm filename VAX[S79,JMC] blob
sn#443113 filedate 1979-05-22 generic text, type C, neo UTF8
COMMENT ⊗ VALID 00027 PAGES
C REC PAGE DESCRIPTION
C00001 00001
C00003 00002 ----Message 1 is----
C00008 00003 ----Message 2 is----
C00019 00004 ----Message 3 is----
C00036 00005 ----Message 4 is----
C00051 00006 ----Message 5 is----
C00064 00007 ----Message 6 is----
C00068 00008 ----Message 7 is----
C00072 00009 ----Message 8 is----
C00074 00010 ----Message 9 is----
C00078 00011 ----Message 10 is----
C00083 00012 ----Message 11 is----
C00086 00013 ----Message 12 is----
C00094 00014 ----Message 13 is----
C00102 00015 ----Message 14 is----
C00106 00016 ----Message 15 is----
C00108 00017 ----Message 16 is----
C00113 00018 ----Message 17 is----
C00148 00019 ----Message 18 is----
C00155 00020 ----Message 19 is----
C00163 00021 ----Message 20 is----
C00174 00022 ----Message 21 is----
C00182 00023 ----Message 22 is----
C00187 00024 ----Message 23 is----
C00190 00025 ----Message 24 is----
C00197 00026 ----Message 25 is----
C00201 00027 ----Message 26 is----
C00202 ENDMK
C⊗;
----Message 1 is----
Date: 19 April 1979
From: Scott Fahlman at CMUA
Subject: Operating Systems for the VAX
It appears that the future computing needs of the major ARPANET laboratories
will not be met by the PDP-10/PDP-20 architecture because of its limited
address space and decreasing commitment to this architecture by DEC. In
recent months, we in the CMU computer science department have been exploring
alternative machines for our basic time-shared computing resource. One
leading candidate for this role is the DEC VAX-11/780. If we and other
ARPANET sites adopt the VAX architecture, we will need an operating system
that is flexible and powerful enough to meet our diverse needs. At present,
we seem to have three choices:
1. Use the DEC-supplied VMS operating system or a modification of it.
2. Use the UNIX system adapted for the VAX by Bell Labs, as is or modified.
3. Produce an entirely new operating system for the VAX.
We at CMU have done a certain amount of digging into both existing VAX
systems. Several people have studied the VMS manuals, many people are
intimately familar with PDP-11 Unix, and implementors of both systems have
visited CMU and answered our questions. We have not developed a unanimous
view about what should be done, but we have gained a lot of information
and some opinions about the advantages and disadvantages of each option.
We want to make this information available to other researchers and, if they
are interested, to the appropriate parties at DEC and Bell Labs. It is of
course to the advantage of all of us to use common operating systems wherever
this can be done without resorting to tools inappropriate to our tasks; in
this way, our work can more easily be shared with other sites.
This file contains several position papers by members of the CMU research
community concerning the choice of an operating system for the VAX, the
reasons for that choice, and what must be done to make the chosen system
usable in the best possible way. Comments and contributions are welcome.
Send them to FAHLMAN @ CMU-10A. From time to time I will add new
contributions to the file and redistribute it to those on the mailing list.
I will try to include only those contributions with some informational
content -- it has been noted that many people on the net have violent
opinions on this matter with very little information behind those opinions.
>> Please use 80 columns across in all responses to maximize readability by
people on narrow terminals. Thank you.
Scott Fahlman (FAHLMAN @ CMU-10A)
---------------------------------------------------------------------------
----Message 2 is----
Date: 19 April 1979
From: McKeown at CMUA
Subject: VMS/UNIX Comparison
SOME COMPARISONS ON VMS AND UNIX FOR THE VAX/780
James Gosling
Steve Shafer
Dave McKeown
April 19, 1979
This is intended to be a first pass at comparing
features and functions available in VAX/VMS and
VAX/UNIX. We expect to expand this document, to
flesh out some of the statements, and add to or modify
our analysis as we receive comments.
All comments are welcome!!!
Vax/VMS features | VAX UNIX features
|
- quotas to restrict resource usage | - no quotas for anything.
no one user can monopolize resources|
- more redundancy in the file system | - fragile file system, but there are
[ as reliable as tops-10 ] | many tools for crash recovery, this
| requires wizardry. Overall, more
| fragile.
- has a more complete set of file |
access protection modes |
- has a TOPS-10 like search list | - search lists are wired into i
| individual programs
- has a generalized asynchronous IO | - all IO is synchronous.
facility. |
- has a generalized demand-paging | - a fairly traditional swapping
virtual memory system. There have | system.
been uncomplimentary comments about |
the scheduler. |
- able to map files into a users |
address space. |
- interprocess communication is more | - interprocess communication limited
elaborate. has named mailboxes. | to simple pipes, both ends of which
| must be inherited.
- Supports DECnet | - supports the Bell internal net;
| code for ARPAnet support exists.
- has a reasonable program sharing | - single segment program sharing,
facility; the library sharing | no subroutine library sharing.
facility is quite deficient though. | ( I mean sharing in the use-the-
| same-physical-copy sense )
- the 'system' language is BLISS-32 | - the 'system' language is C
- the OS is written almost entirely in| - the OS is written almost entirely
assembler & is hence very difficult | in C which makes it very easy to
to modify. The mere mass of the | modify. The entire Unix kernel is
code makes it difficult to | very small, clean and easy to
understand. | understand. Relative novices can
| work in the bowels of the OS with
| little time spent learning how it
| it works.
- kernel calls are baroque & difficult| - there are very few kernel calls,
to use; they use complicated control| all taking a small number of simple
blocks and calling sequences. | arguments.
- the command interpreter is quite | - the command interpreter is quite
conventional. | sophisticated; it has facilities
| to interconnect programs via pipes
| and execute them in the background
| in a natural manner.
- wild card file names are special | - wild card conventions are
cased on a program-by-program basis | uniformly applied by the command
| interpreter.
- very little existing software; | - there is a vast body of existing
essentially just that from DEC. | software for Unix, most of which
| is trivially transportable to VAX.
| [ notably: many editors, compilers,
| compiler generators, document
| production programs, the list is
| long ]
- the manual is distinct from the on- | - the manual and the on-line help are
line help. Many feel that the | one and the same. It is quite
manual is overly verbose. Copies of| concise & generally clear. This
the multi-volume manual must be | arises primarily from tight
obtained from DEC. | standards & a clean system. Since
| the manual is online, we can produce,
| and have produced, copies locally.
- the file system is hierarchical with| - the file system is hierarchical
many different file access methods, | only one file access method: random
but this too has a baroque and hard-| access byte stream; it is simple
to-use flavor. | and easy to use.
--------------------------------------+--------------------------------------
Anything that one can do in Unix one can do in VMS, while the
converse is not true; there are things which can be done in VMS
which cannot be done in Unix. But there is a very clear tendancy
for VMS to be very intricate and hard to understand, it has many
features that result from a desire to be compatible with other
systems and from an IBM-like desire to be all things to all
customers. In many cases the same feature is repeated, but in a
slightly different flavor. On the other hand, Unix is
extraordinarily simple with a small number of orthogonal
features. It does not attempt to cover everything that everyone
would like to do, it has sacrificed functionality for simplicity.
The point about the language in which the OS kernel is
implemented is significant. The fact that the Unix kernel
is small & written in a high level language means that it is easy
to understand and modify; while on the other hand VMS is written
in assembler, which along with the complexity of the OS would
make it difficult to either understand or modify. Many of the
features that are missing from Unix are essentially trivial to
add. Witness the immense number of mods that have been made to
the IUS and SUS Unix systems.
-----------------------------------------------------------------
----Message 3 is----
Date: 20 April 1979
From: Scott Fahlman @ CMUA
Subject: Notes on VAX/VMS and VAX/UNIX
This is my own personal assessment of the relative advantages of the two
existing operating systems for the VAX: DEC's VMS system and Bell Labs'
UNIX. Note that this is based entirely on manuals and on talking to the
implementors of the systems; I have not had hands-on experience with
either system on the VAX and have not used PDP-11 UNIX enough to be
really proficient with it.
Let me state my conclusions at the outset, then give my reasons. I believe
that both VAX UNIX and VAX VMS are inadequate for a research environment
like CMU/CSD as they stand, but that either system could be made truly livable
(comparable to the best systems now available and better in some important
ways) with a reasonable amount of effort on the part of the research
establishments concerned. By "reasonable", I mean an effort far smaller
than would be required to produce an entirely new system. I see no need for
such a third-system effort at present.
Of the two systems, I prefer VMS as the starting point because it is
significantly stronger in fundamental internal strucuture. The VAX UNIX
system offers several advantages, but these advantages are mostly superficial
and would disappear in the course of time. Basic design flaws will be with
us for years to come.
FUNDAMENTAL DIFFERENCES:
These are things with roots deep in the bowels of the respective systems,
and which it would require a major effort to change.
- File system reliability: VMS is considerably better than UNIX here.
- Overlapped asychronous I/O: VMS has it, UNIX doesn't.
- Large address space virtual memory system: This is the principal reason to
move from PDP-10 or PDP-11 to VAX, and is therefore of critical importance.
VMS has virtual memory designed in from the start; UNIX promises to add it
soon. I suspect that the fact that UNIX was developed for a machine with
a very small, non-virtual address space has had rather pervasive effects on
how things are done in UNIX, and that it will be quite a while before the
UNIX environment as a whole takes full advantage of the VAX's memory
architecture.
- Interprocess communication: The VMS scheme, which provides sharable data
segments, sharable event flags, and a very general general mailbox
scheme seems considerably more powerful and flexible than the UNIX scheme
which provides only fixed byte-pipes between processes.
- System maintainability: This is the only fundamental area in which I see
UNIX as the clear winner. UNIX is a small, simple system implemented in
C; VMS is larger and more complex (a result of some of the features noted
above) and is written mostly in assembly language. (DEC had BLISS-32
for system programming, but blew the opportunity.) This is unfortunate for
potential users of VMS, but I don't see it as fatal. It might be worthwhile
for DEC to mount a project to re-do VMS or substantial parts of it in
BLISS-32, but I doubt that this will happen.
- Vendor co-operation: CMU has apparently had no trouble with Bell Labs, but
there are some disturbing stories of people being screwed by Bell's standard
licensing agreement which treats UNIX as a trade secret. It is rumored that
IBM will not hire ex-UNIX hackers for fear of being sued for theft of
secrets. If true, I find this scary. No such nonsense surrounds the
DEC software. DEC seems to be eager to cooperate with CMU in taming VMS
for use in our sort of environment. I don't know what Bell's attitude is.
SUPERFICIAL ISSUES:
These are fixable with at most a few man-months of effort.
- Command interpreter: The UNIX system's command interpreter is powerful,
flexible, and easy to extend. The VMS command interpreter is a disaster.
It does not provide the user with the ability to run several jobs at once,
or even with the ability to suspend a process, run something else, then
resume the first process. It is inadequate in many other ways. It is
this turkey, astride the principal user interface, that is responsible for
the growing reputation of VMS in the ARPANET community as a useless system
for research.
If we go with VMS we will have to throw this module out and completely
redo it. This should not be too hard, since the VMS system explicitly
allows for user-defined command interpreters and the basic system calls
that do what we need to do are present in the system. I estimate that
with a few man-months of work we could produce a command interpreter that
combines the best features of UNIX, TOPS-20, and ITS, and is easily
extendable. It would be written in a high-level language, probably BLISS.
- Simplicity of system interface: UNIX provides a simple, fairly elegant
system interface to the programmer. VMS is hairy. Some of this hair
is necessary to provide the additional useful features of VMS; most of it
results from the inclusion of features that are useless to us; some is just
poor design. In any event, we can hide most of this complexity from the
user by building up a library of subroutines and macros to do the things
we want to do. The uniform subroutine-calling convention of the VAX
should help here.
- Availability of software: Right now, there is a lot of software for PDP-11
UNIX which can be moved to the VAX. At present there is little software
for the VAX/VMS that would be useful to us. This will change very quickly
if we and other ARPANET sites go to VMS. Much of the UNIX software could
be used on VMS once a C compiler is adapted for that system and a few
utility programs are built. This applies only to software in the public
domain and that which we could license from Bell, but that should cover
most of the useful programs.
- On-Line documentation: UNIX has this; VMS has only hard-copy manuals.
There is no reason this feature cannot (gradually) be added to VMS. The
ZOG group at CMU plans to implement a ZOG-net manual for VAX VMS if we
end up going the VMS route. This should be at least as good as the UNIX
manual, probably better.
- File format: UNIX has a simple and elegant file format. Every UNIX file
is a randomly addressable array of bytes. VMS provides a comparable
facility with the added ability to map a file into a process's address
space. In addition, VMS provides a record management system (RMS) for
data-base flavored applications. RMS provides many features that the
COBOL users think they want, and the interface to it is hairy.
Unfortunately, some clever soul at DEC decided that RMS was a good way
to provide programs with device-independent file interface, and most
of the DEC software sits on top of RMS instead of on the basic file system.
ASCII files, for instance, do not use newline characters but instead
store each line in a record. The line editor builds such files; the DEC
compilers expect them. This use of RMS certainly makes life difficult for
programmers and is reputed to be grossly inefficient. We will probably
want to undo this mistake. This means making relatively simple changes
to the file interfaces in the DEC software we use often (we will have to
have the sources for this) and building an interface program to connect
non-RMS files to the DEC software we haven't fixed. Or else we must
convince DEC of the error of their ways and get them to build software
that will work with simple-format files as well as with RMS.
TAMING VAX/VMS:
This is a list of the changes and additions that, in my opinion, we will
want to make to VAX/VMS if we adopt it as our principal departmental
time-sharing system. Some of these have been mentioned already. If
some of these items are done at sites other than CMU, so much the better.
- Build an entirely new system command interpreter, in which users' jobs
are run as independent sub-processes, and in which several of these can
be run concurrently. The command language must be well-designed and easily
extendable.
- Modify the DEC file protection scheme to allow users to be members of
multiple overlapping groups. The current scheme in which each user is
in only one group does not reflect our environment. The overlapping
protection groups, along with the current VMS ability to share
sub-directories, should give us a very flexible system for multi-person
projects. This should be a relatively minor change. Each user's
authorization file will contain a list of groups instead of just one,
and when the user tries to open a file he succeeds if any of his groups
gives him access. The user will not have to explicitly change groups
to do this.
- Eliminate the arbitrary limit of 8 levels of directory. If this can't be
made infinite, at least make it large (32 or 64 levels). This should just
require that we enlarge some tables and reassemble.
- Do whatever is necessary to remove RMS from the guts of the system. Keep
it around as an optional package for the tasks in which it is truly useful.
Users who are not doing data-base aplications should never have to know
about RMS, whether it is actually gone or is hidden under a clean interface.
- There is some suspicion that the DEC scheduling and swapping algorithms
may not be optimal for our needs. After some experience with this system,
we may want to tune or redesign these. The situation is unlikely to be so
bad that we will have to do this right away, and we may be pleasantly
surprised.
- Write lots of software. We need SCRIBE (already present), a screen editor
(I'm doing this now), a mail system (can be integrated with the editor),
terminal links so that logged-in users can type at one another, an Ethernet
interface, code to deal with an external file server, code to support
high-resolution bit-mappped displays, a LISP (being built at MIT), assorted
subroutine libraries, and lots of other things. Most of this same software
would have to be written for UNIX.
-----------------------------------------------------------------------------
----Message 4 is----
Date: 23 April 1979
From: Joseph Newcomer at CMUA
Subject: Some Reactions on the VMS/UNIX Comparisons
Up to and including the observation of the "system language" VMS looks like a
clear winner over UNIX. Therefore, I'd like to spend some time covering the
other points:
- Kernel calls: DEC, as usual, wrote an operating system interface for
assembly code programmers; this is unfortunate, especially going into
the 1980's. However, if we do it right, the operating system
interface could be made reasonable for BLISS. However, it is
important that if the BLISS-32 library does not come with a complete
operating system interface, that WE DO ONE RIGHT AWAY, and not let it
"grow" like the BLISS-10 and SAIL libraries did! The effect of
uncontrolled random growth is absolute chaos.
- Unix clearly did the right thing about operating system interface at
the C level: every operating system call is just a procedure call.
However, I understand that PDP-11 Unix requires a separate,
assembly-code, interface between C and the operating system, as the
operating system returns status in the condition codes, a feature C
does not support. This does seem rather silly. The operating system
interface should not have to be written in assembly code!
- (DEC has a propensity in PDP-10 software to "overload" the UUOs,
i.e., depending upon what the parameter list looks like, the UUO can
do one of several things. This is a real pain when dealing with a
high-level strongly typed language such as Algol-68, or even weakly
typed languages like Pascal. One wants to use the type system to
prevent the programmer from doing absurd things, such as adding
channel numbers to pointers to obtain floating point numbers...) The
design of a really good high-level interface should also be language
independent, in the sense that the number of parameters and their
order, and the name of the operation, should be identical in BLISS,
SAIL, C, FORTRAN, Ada, etc. Whatever we do must meet this criterion,
if we don't want to repeat the mistakes of our own history.
- Command interpreter: Rick Snodgrass has written an interpretive
system, COLA, based on the Smalltalk paradigm. It is currently
running under Hydra and is written in BLISS-11. Although the current
implementation is slow, it has not had much attention paid to
optimization at all; I expect we could do better as we put more
effort into it. In terms of power and flexibility, it is superior to
the VMS interpreter and probably to the UNIX interpreter. We are
giving it serious consideration as a new shell for VMS.
- UNIX clearly did wildcards right.
- What prevents C code from running under VAX/VMS? Certainly not
licensing agreements---as far as I understand it, if we purchase an
academic license to run UNIX on our Vaxen, what we have really
purchased is the right to run certain systems written by Bell Labs on
particular processors. I don't think that it matters if the C code
runs over VMS or UNIX. If we had a C compiler which ran under VMS, I
suspect that the large body of UNIX software (which I find
disappointing in terms of either human or software engineering) could
be transferred by simply defining a run-time system that interfaced
to VMS. This possibility is worth investigating.
- I find UNIX on-line help concise to the point of obfuscation. DEC
documentation, on the other hand, is verbose beyond the limits of
comprehensibility. DEC needs a team of good technical writers.
Their ability to create an index, for example, borders on the
incompetent. However, my ability to locate items in the UNIX manual
is limited by the fact that I don't know that program "A" and program
"B" are really different manifestations of "Q", so I can't find them
under "A" or "B" in the manual; only under "Q". While on-line help
is nice in Unix, when there are more than a couple users on the
system I can search hardcopy faster than the system can respond to a
search request. I therefore don't think the difference in
documentation is compelling; after all, we can type in the relevant
parts of VMS documentation.
- A simple and easy to use file system appears attractive, but I'm not
sure that restricting oneself to a simple subset of VMS files does
not result in the same simplicity. Of course, there is always the
temptation to use something more elaborate...
- I hope, but don't really believe, that if we wanted to rewrite a
module of VMS we could do so in BLISS-32 and be able to interface it
directly with VMS. In this way we could gradually replace the worst
aspects of VMS, such as the scheduler is alleged to be. Whether we
choose VAX or UNIX, we might as well resign ourselves to working in
the operating system; UNIX is a winner by its simplicity, but of
course that also means limited functionality.
- I have heard that the PDP-11 UNIX operating system is overly
simplistic, not very modular, and hard to make substantive changes
because of undocumented interactions among components. Since I have
not worked in UNIX, I can only offer these as "rumors on good
authority" and let those who have worked in it respond.
- I also understand that it is substantially easier to add new device
drivers to VMS than to UNIX; in particular, it seems that VMS device
drivers can be developed more or less as user code, and linked in
"dynamically" by setting up pointers within VMS.
- There is rumored to be a UNIX system which runs under (over?) VMS; I
think we need data on this system before we can make a choice.
- I should point out that a file system which involves wizardry to
maintain is suspect; only in extreme cases does Howard have to patch
the PDP-10 disks. My current understanding is that a crash in UNIX
requires wizardry to bring up the system again; I was an unfortunate
victim of this, losing five hours work because the system crashed as
I was editing a file. While I have lost files on the PDP-10, I have
never lost one because of a coincidence of editing and a system
crash.
- Overall, I suspect that I could live with either system for a year or
so. However, I think it is important that if we choose Unix,
BLISS-32 be available under UNIX (the same reasoning applies to this
as running C under VMS...as long as we have the license, the
operating system should not matter). If we can't run BLISS-32 under
UNIX (but can run C under VMS) then UNIX is unacceptable. Not only
are there good reasons for having a decent compiler, but we should
have a common language between our two major machines (PDP-10 & VAX).
Common BLISS will allow us to make portable systems for both the
PDP-10 and VAX. The KL10 will still be with us for a number of
years.
- Note also that if we get two Vaxen, we could run UNIX on one and VMS
on the other, as long as we could transfer C and BLISS-32 source
portably between machines. This would at least leave our options
open, albeit at some pain in moving systems around.
- As a meta-issue, there is a lot more commitment from DEC to maintain
VAX than there is from Bell to maintain Unix. While it is nice to
believe that a friendly, helpful Unix community will be able to share
code, in practice major operating system enhancement has too many
dependencies on prior work [I've worked in enough such systems to
know]; thus, if somebody rewrites a chunk of Unix we may have to
spend a lot of time figuring out how it fits into our heavily
modified system. DEC has to support a lot of VMS implementations,
and will be producing new releases at reasonable intervals. Again,
if they are reasonable, it is likely that any BLISS-32 modules we
would rewrite in some significant way would go back into the system.
Where will we be in five years with Unix?
- <<I had in here a highly insulting attack on people who would, in
this era, even propose writing a major operating system in assembly
code, but I have decided to censor it. -jmn>>
--------------------------------------------------------------------------
----Message 5 is----
Date: 1 May 1979
From: McKeown at CMUB
Subject: Reply to Fahlman on VAX
Some Comments on Fahlman's Notes on OS for VAX
James Gosling @ cmua
Steven Shafer @ cmua
Dave McKeown @ cmub
May 1, 1979
Please note that we speak from a position of considerable familiarity
with the internals of Unix, and little familiarity with VMS.
FUNDAMENTAL DIFFERENCES
- file system reliability: This is not fundamental, it is superficial,
users never see it. We assert that with very little effort
(2-3 man-months) the Unix file system could be made
much more robust. At present it is not dramatically more
fragile than tops-10, it's biggest problem is a tendancy
to damage the files that were being written to at the time of a
crash. Current mean time between failures on UNIX are on the
order of 60 hours. Every crash does not imply significant
file damage. In fact, when possible, IUS reboots itself, cleans
up the file system, and carries on without any intervention.
- Overlapped asynchronous I/O: currently Unix does this transparently
for the user in its own hidden buffers. Its algorithms are quite
effective. As far as allowing the user to explicitly do overlapped
IO, let's ask the following questions: In how many cases would it be
superior to internal Unix buffering? How often do you need or even
want to spend the time to do it right? Any library-type async-IO
routines are unlikely to be superior to those in the monitor, the
only times that one can out-perform the monitor is if the user has
some special knowledge about the access patterns of the file and
hence can do a better prediction job or when the monitors buffers
become full.
In any case, while Unix does not allow async-IO to directory-
structured devices, it does allow it to what are known as "raw" devices,
these include the magtape, unstructured disks, and appropriate
communication lines. There is just a separate set of IO system calls
which when used with certain files causes the IO to happen
asyncronously. We have implemented such a set of routines on
the IUS/UNIX machine.
- Large address space: it is NOT the case that the small virtual address
space of Unix processes has had a pervasive effect on how things are done.
The C runtime system encourages a programmer to think that he is
programming in an essentialy infinite environment -- things only break
if the storage limit happens to be exceeded. The number of programs
whose design was influenced by the fact that a limited amount of
storage was available is small, few programs actually have any
use for more than 64Kb of storage. However, for those that do
need more, PDP11 Unix is a disaster; witness INGRES, a large
relational data-base system for Unix. However, that is an
architectural artifact of 11/UNIX. The VAX implementation
of UNIX has many of the hooks for scatter loading in the right
places (we are told by Bell) to implement virtual address
capability. It would be interesting to see what the performance
issues (other than being unable to map files to user address space)
really are. What are these pervasive effects? There may be some that
we haven't thought of, so let's try to identify them and see if they
really pose limitations.
- interprocess communication: agreed, stock Unix is quite deficient. Various
installations have fixed many of the problems and their modifications to
Unix could be obtained. However, pipes have some generality over
mailboxes, they are deficient in that they are not "named". What
is "fixed" about UNIX pipes?
- system maintainability: We agree with Scott here, but he seems to
contradict himself later on -- admitting that VMS would need
considerable work to make it livable while admitting that VMS is
a pain to modify. We envision major mods to VMS or UNIX before
we would suggest that they were usable to the Dept., certainly
over time more modifications will be necessary. It should not
be forgotten that we will have to live with a bad choice
of implementation languages for a long time.
- Vendor co-operation: IBM scare tactics should be ignored; to use
this as a factor would be ridiculous. Can we put the IBM/UNIX
business to rest? This is not a fair argument unless it
can be substantiated. We admit, Bell's attitude
is reasonably indifferent, but the fact is that we have not
had to rely on Bell for any support. DEC TOPS-10 support
has exhibited a fairly low vapor pressure around our machine room.
Let's face it, either way, we roll-our-own.
SUPERFICIAL (sic) ISSUES
(We assume here that these superficial issues will EACH take
a few man-months of effort)
- Command interpreter: reasonable, except that from my reading of the
VMS manual, such things as pipes & multiple processes are quite
difficult to handle cleanly.
- simplicity of system interface: agreed, you can paint over almost
any atrocity. Multiple layers of "subroutine libraries and
macros" leave something to be desired, even if it is done "right".
This is a major issue.
- availability of software: again agreed, given that you can paint over
any atrocity, you can move any program anywhere. But at what cost?
If the ARPA community goes UNIX, much of the non C software
could be adopted. There is no reason that WE know of why a
BLISS 32 compiler would not run under UNIX.
- On-line documentation: *** This is neither a superficial issue nor can it
be fixed "with at most a few-man-months of effort" *** To do this right
would require rationalizing and standardizing the manual, hiring a zillion
secretaries & committing both ourselves and DEC to the maintenance of this
manual [ the committment is already made for Unix ]
- File format: How often does one need fancy file formats? Can you live with
the set that VMS gives you? Or would you have to write your own anyway?
Needless to say, we don't write COBOL programs and the programs that we
write are unlikely to fit into the COBOL model of the universe.
Scott speaks of 'undo[ing] this mistake' by 'making relatively simple
changes'. I simply don't believe that it's possible to do with a
reasonable amount of effort. This is a major change.
*******************************************************************************
It appears to us that most of the complaints that people have about Unix
concern features that are invisible to the average user. They can be changed
with few adverse affects to the user community, they generally do not involve
changing existing features in the abstract machine seen by the user.
A good case in point is the UNIX file system vs. RMS. The
deficienties in robustness of the UNIX file system can be handled
without the user, or much of the operating system knowing about it.
The interface is clean, simple to understand, (undergrads study it
by reading the code as a case study in many universities)
and simple to modify. RMS is truly a bear. We need to rewrite
RMS, (in assembler?), no!! we need to redesign it, and possibly
(probably) change all VMS programs file handling routines. As
an example, the bogus text-file format is all too pervasive.
It is clear to us that these point by point discussions
shed some light on the type of problems that will be faced given
that either operating system is selected. It is difficult to
really quantify the amount of work necessary to make these
systems both robust and habitable. Suffice to say, neither
system will have many of the features we will want 5 years
from now. The question really should be how hard is it to
mold each one into some version of what we can live with
down the road. We feel that UNIX has several advantages
in terms of conciseness and readability of code, and the
lack of major design efforts to beat reason into mistakes.
-----------------------------------------------------------------
----Message 6 is----
Date: 2 May 1979
From: Ron Brender at CMUA
Subject: Reply to Fahlman on RMS
Scott,
I have looked over the VAX.TXT file and see no reason why it should not
be opened up to other comments. As neither a UNIX nor VMS user myself
it is hard for me to add much to the discussion, with one exception.
I would urge great caution and deliberation in the obviously strong
sentiment for throwing out RMS and rolling your own. What ever you
feel about its technical merits the following facts are important:
1) It is THE DEC supported I/O interface, and will be the basis
for future enhancement and development. New capabilities will
be offered as an integral part of it.
2) It is the most heavily used of the system interfaces on VAX
(second is the CLI and third the exec calls I would guess).
3) In particular, it is the interface used by the run time systems
for DEC's languages. You may not care much for FORTRAN and
COBOL, but when AND IF languages like PL/I, C, PASCAL, ADA
are offered by DEC they will use this interface. Your favorite
language from elsewhere will either use this interface
or something "special" to its origin and thus be hard
to adapt.
If you want something simpler, than I suggest you consider building
on top of RMS rather than beside it. The XPORT interface for transportable
I/O might be considered as a candidate in this regard. (There are a couple
of copies of preliminary specs in the department (eg. Newcomer, Lamb,
and other I don't recall at the moment).)
If you are worried about performance with RMS I suggest you first
measure and demonstrate that RMS is a (the?) culprit and that you
will really do better.
YOU MAY BE RIGHT ABOUT RMS, BUT I THINK YOU HAVE YET TO PROVE YOUR
CASE. Be careful on this one in particular!
Ron
----------------------------------------------------------------------
----Message 7 is----
Date: 2 May 1979
From: Scott Fahlman at CMUA
Subject: Reply to Ron Brender
Ron,
It is clear that we don't want to throw RMS away. If I seemed to be
advocating that, I recant. Much software will expect it, some
parts of RMS (the stuff for opening files) is universally useful,
and the rest is useful for doing-data base things. My only objection
is that TEXT files have to go through RMS and be chopped into lines.
That's fine for SOS and assemblers, but lousy for many other things.
In a screen editor, for instance, it might be vastly more efficient to
simply store the text as bytes and map the file into virtual memory.
It seems a shame to pay the overhead for RMS if it is making life
difficult rather than easy for the applications that run most often.
And the cost is not only in runtime, but also in cognitive load on the
user. Everything ends up being as complicated to do as the hairy
data-base applications that make real use of this stuff. Unix did
this right: simple files are simple, and you can add hairy formatting
stuff if you like.
The simple fix is to write a bunch of library routines to hide all the
ugliness, and make the editor speak RMS like everything else -- it could
edit the files in raw format and stuff the lines into records as a post
pass. Or we could cut deeper, store the files in raw format, and get
the rest of the system to deal with such files. Things we write here
would be easier to do this way -- they wouldn't have to deal with RMS --
and things we get from DEC could either have their readers fixed or,
for things we don't use often enough to modify,
could suck input through a pipe-process (provided as a standard utility)
which makes raw files look like RMS files. This latter scheme would
only be preferable if RMS really is as inefficient as rumored. I have
heard that the penalty is a factor of two or three in time spent doing
I/O, but you are right -- that does need to be verified.
It is really unfortunate that DEC didn't separate the concept of
device-independent I/O of simple byte-array files from the package
for turning these simple-format files into hairy records, but such is
life. We can live with it, but UNIX gets a few bonus points on this
issue.
-- Scott
---------------------------------------------------------------------
----Message 8 is----
Date: 3 May 1979
From: Brian Reid at CMUA
Subject: Comment on RMS for VAX/VMS
RMS imposes a record-oriented view of files on the unwilling user, which
is a restriction that I, for one, would not be willing to live with.
It is also incredibly complicated. I have spent hours and hours poring
through the reference manuals for it, and it is definitely the product
of the same sort of mind that brought us flying buttresses and DATE75.
It attempts to provide universality by a combination of unwillingly
allied weak concepts rather than some single operational concept strong
enough to support everything.
Preliminary results by the Chemistry people have shown that RMS costs
a factor of 10 to 20 in program speed over doing the same I/O by hand.
----------------------------------------------------------------------
----Message 9 is----
Date: 2 May 1979
From: David Smith at CMUA
Subject: SOME COMMENTS ON VAX UNIX
Judging from John Reiser's comments, we can expect that Bell
will not put virtual memory into VAX Unix. He claims, however,
that a knowledgable person could add it without too much trouble,
as the scatter-loading and partial-swapping features do most of
the necessary work.
I suspect that virtual I/O (file mapping) will be more difficult.
Might it induce a circular dependency between the pager and the
file system? Any given page might have to be brought in from a
different file. Since the bare page tables take 1/128 of the process'
space, pager information on a per-page basis might get expensive.
To keep expected costs down, we might be forced to add some hair.
On the other hand, VMS must run into the same problem; maybe this
is a source of some of its hair.
To my knowledge, the only place where the small address space
assumption is wired into Unix (and maybe this is a slight misuse
of the term "wired in") is the "fork" call. "Fork" generates a
child process which is identical to the parent except for a status
bit returned to each so it can identify itself as parent or child.
Usually (after possibly redirecting I/O), the child process
immediately EXECutes another program. This means that its image is
replaced by the contents of the new program's object file, thus
destroying the newly-made copy of the parent. This
procedure may be OK on the PDP-11 which places a small maximum on the
size of the copied process image. But on the VAX, the amount of
useless copying could be very large. Of course, most processes
really are small. Also the text pages of parent and child can be
shared. And it could be possible to just copy the page
table, and set the page status to "make separate copy when one of
the processes attempts to write". But if Unix had first been designed
for a machine with a large address space, the forking primitive would
probably have been designed differently.
---------------------------------------------------------------------
----Message 10 is----
Date: 5 May 1979
From: JNC @ MIT-AI (J. Noel Chiappa)
Subject: VAX Operating Systems
Well, I am not a heavy VAX user, but I have done some
work on it, and I am a UNIX hacker. As to IPC on UNIX, there
are various home-rolled alternatives, e.g. RAND ports, which are
as general as mailboxes. As to the FS, the last time I had a system
crash that left the disks in a wedged state (requiring stomping
on bytes by hand) was months ago, and I run an experimantal system
that is always in yoyo mode. The supplied salvager is adequate to
deal with almost all the common things that happen. Also note that
VAX/VMS does not do page repatriation a la MULTICS, so if your system
crashes (or so says our VMS hacker) you lose. UNIX does do repatriation
(in a user daemon process which is part of the default startup.)
UNIX is orders of magnitude easier to modify internally. Device
drivers for UNIX are as easy to do as VMS for simple devices,
and far far easier for complicated ones. I speak from bitter
experience here - our high speed local network interface
for UNIX I essentially did in one weekend, while its VAX/VMS
counterpart took months. AS to asynchronous I/O, the other hackers
already said some of what I would. I have also added devices to UNIX
that did asynchronous I/O (explicitly - you get an interrupt
when I/O is complete, a la VAX AST) and it would be easy to add if
anyone had any use for it.
This is all beside the point, which is that UNIX
is about 1000 times easier and nicer to use. I doubt that ANY
command interpreter on VAX could function the way the UNIX
one can with respect to I/O redirection. The doumentation
is in a different class altogether (emminently readable, as
opposed to hopelessly complicated). The system internals are (again)
in a different class - having looked at both at length I can
assure you that a UNIX with paging would STILL be far simpler
and easy to read.
I you were to throw away RMS, CLI and most of the other user
level stuff, the what was left of the VMS kernel is passable. The UNIX
file system orginization is much nicer, with directories being treated
in a much more natural manner. In my mind, you'd
be better off fixing the things in UNIX that are built around
a small non paging machine than rewriting in a wholesale
fashion VMS.
I will refer you to the DSSR group at the Lab here, who have
had UNIX (11/70 3/4Meg 600M disk) for a while and just got a VAX,
on which thay now have C and a working LISP. They plan to punt VMS
in favour of UNIX ASAP.
-- Noel
-----------------------------------------------------------------------
----Message 11 is----
Date: 5 May 1979
From: WNJ at MIT-AI (William N. Joy, U. C. Berkeley)
Subject: VAX UNIX
You should really talk to us folks here at Berkeley about VAX operating
systems. We have experience with both VMS and VAX/UNIX (which we were
the first to run outside of Bell Labs). We are putting VM into vax/unix
and have a lisp system, screen editor, etc already running.
There are a number of misconceptions in the file you have on VAX Systems.
To wit:
1. The new version of UNIX does not have a fragile file system. We never
have any trouble.
2. Search lists are not wired in, rather are part of the ENVIRONMENT
(a new concept in version 7 UNIX).
3. A paged version of vax/unix will be running here with simple sharing soon.
The form of desirable sharing in a vax-like architecture is under discussion;
VMS certainly doesn't have the answer.
4. There is a decnet driver for UNIX readily available (written byy Jim Hamilton
of DEC). We have a version running between an 11/70 and a vax (running vms)
here over KMC-11's.
5. VMS drivers are difficult to add and a mess, as is the entire system.
We have added several drivers to UNIX easily. Looking at a VMS driver is a
waste of time.
-----------------------------------------------------------------------------
----Message 12 is----
Date: 5 May 1979
From: RJF @ MIT-MC (Richard J Fateman, U. C. Berkeley)
Subject: VAX Operating Systems
We at Univ. California, Berkeley, have been using VAX 11/780 since October,
1978, and have used VMS and (mostly) UNIX.
(1) There is an extensive teleconference sponsored by Dept. of Energy
on VAX VMS/UNIX which you must see before continuing further.
Ask POOL@BBNB for entry if you do not have access now.
Especially interesting may be the description of efforts at LBL to make
vms look reasonable (in the LBL context, read "like unix").
(2) Peter Weiner's Interactive Systems has a "pseudo-unix"
on top of vms. -- if you want one for $100,00 for each cpu -- and then it
doesn't do what you want, I think, but you can be the judge. I.S. is in
San Diego.
(3) BLISS-32 (vms) binary costs $10,000 (per cpu). If you think that
BLISS-32 is a higher-level language, first look at how they've used it.
(4) VAX VMS machine readable source is $10,000 (per cpu). I have
a copy. It is 5 2400 ft mag tapes.
If you think DEC will support your changes to VMS, lots of luck.
(DOE teleconference has some info on this.) Would you like to have
a modified ibm 360 os/mft (e.g.) as a basis for modifications? You
may be building on a potential dinosaur when/if DEC writes
VMS version xxx.0 in BLISS-32.
(5) You will certainly be able to run C programs on VMS, as has already
been done. That doesn't mean you can run all of UNIX.
(6) The screen-oriented editor, EX, developed here at Berkeley is
likely to be the equal of any such key-board + screen editor around.
It has been modelled after EMACS, but deals with more terminals better,
especially at lower speeds. At 9600 baud it is of course better yet...
It has lisp mode; tags; "undo"; recovery of files from system crash
to within a few commands; terminal capabilities file: from a
tty-33 to a c-100, hp 2645, ...; interfaced to mailer which uses
local network. Mailer has features of TENEX sndmsg, MIT-ITS rmail...
I know it is impossible to discourage people from writing code on VMS,
but it makes them loyal without >>good<< reason, Scott.
(7) We have lisp (since November), essentially Maclisp compatible but no
compiler yet. It is written entirely in c except for
bignumbers. It supports MACSYMA subject to pageing availability. MIT's lisp
(NIL) when available will be "system independent", hence will have to
run under UNIX...
(8) We have DECNET written in C for unix though we of course
do not use it for our local net, now connecting 7 11/70's and 1 VAX
(soon to be enlarged to include arpnet).
(9) Paging has been incorporated into vax/unix kernel; as Reiser said,
not much sweat. We expect to release to vax/unix licensees in late June.
It does not mung up the system. As for small-address dependencies, there can
hardly be any, since we run jobs much larger than 64kbytes already.
(Though more than 440kbytes on a .5 meg machine is the limit without paging
running).
(10) We have hard evidence to show that writing a driver for vms is harder
than for unix, since we have written several for unix which dec hasn't
managed for vms. The dynamic reconfiguration is largely a red herring.
once you get a disk driver working, you don't change it often.
We already support unibus disks (non-dec) and a versatec plotter-printer.
DEC would like to know how to do those things.
(11) FORTRAN bencmarks suggested that VMS is more efficient than unix (30%)
for random access file read/write. I do not know the details of this.
I have not seen such difficulty with the unix file system vis-a-vis INGRES;
Maybe I just don't know the problems with RMS first hand; all
I've done is put the RMS documents on a shelf.
(12) The UNIX/IBM story: I have copies of the relevant letters, and I think
the IBM lawyers have taken a foolish position and MIT has (intially)
over-reacted. My advice (I am not a laywer, however) is to ignore it.
(13) Programs on our vax unix provide extensive and quick
cross refeences to system and user programs via an "apropos" command.
I have been told that manual access can be speeded up at the cost of more
disk storage. Unix documentation seems obscure only until you understand the
brevity conventions. It can be a bit startling to see how short something
can be.
(14) Bugs: we have found a few in unix, and had them fixed at Bell (or
locally). Mean time between software failures is probably on the order
of 1 week or more.
(15) I do not know of any significant software written for VAX in BLISS-32;
The fortran compiler is written in BLISS-11. It is not true to say that DEC
has no VAX software. It has lots. Unfortunately it is mostly PDP-11 RSX-11
stuff.
I suggest that before you guys invest too much more time on this, you
either visit Berkeley, or invite some of us out there. I will pass your
file on to unix wizards etc here for further comment.
Good Luck
-- Richard Fateman
---------------------------------------------------------------------------
----Message 13 is----
Date: 6 May 1979
From: RMS @ MIT-AI (Richard M. Stallman)
Subject: Comments on VMS
1) VMS attempts to provide the feature of mapping files into
the address space, but blows it so badly that one of the main applications
is completely screwed.
They thought it would be useful to give the "section", which is a part
of a file that you can insert in your address space, a name of its own
distinct from a filename. This of course is worth absolutely nothing.
And it creates a marvelous screw. The section name space has to be protected.
Anybody can make his own private section names, but it takes privileges to
create a global one. If two jobs individually make private sections that
overlap, from what I hear the system does not know what is going on, so if
they try to modify the file they get out of synch. So, there is no way at all
that two people with no privileges can decide to run programs that will share
memory which they can write in.
2) User-visible asynchronous IO is useless on a timesharing system.
Assuming that there are a reasonable number of users, each job
might as well just wait until its IO is completed - someone else
will use the time. What is necessary is the ability to ask whether,
or be notified when, input or buffer space is available. Providing
user-visible asynchronous IO provides this, but in such a way as
to require the user to worry explicitly about double-buffering,
which is just a nuisance. IO which is synchronous as the user sees it
with optional interrupts is very simple for most applications, as well
as providing the buffering in the monitor, and for hairy things it is
better as well, because the user can still count on being able to re-use
his own buffers as soon as the IO system call returns.
3) Another thing about VMS: hardly any of the important operations
for controlling subprocesses exist. I spoke to the DEC person who
designed the system, and he says that they think those operations
should not exist, that they are dangerous, and can't imagine
a mode of operation in which they are useful. My invitation for him
to come and see one was declined.
Please pass this on. I have tried to make things brief rather than
rigorous, so anyone who isn't convinced should ask me for a detailed
explanation before commenting publicly.
You should look at RMS;VAX LOSSES.
<< Following is the file referred to. It was created 10/28/78 for
another purpose, but raises important questions about sharing,
process control, and other problems with VMS. -- SEF >>
VAX LOSSES:
1) Can't create a procedure suspended.
Can't read or write its memory.
Can't give it logical names.
Can't assign it channels.
Can't cancel its io.
Can't map its memory or map memory into it.
2) Too many things are only per-process or per-group.
You can't do something that is per-user or per-tree.
Things that you would want that way are named sections
and logocal names.
3) All processing of logical names should be recursive.
The winning underscore convention should be used when you
want to stop the recursion when it would otherwise continue.
The crazy command interpreter escape headers should not exist
or should be handled automatically by all system services.
4) It is a loss that most people can't create system global sections.
It should be possible to make a sharable library just by writing
the file. A temporary system global section should be made
automatically when relevant.
Also, this makes it hard to write a program which has the
property that anyone can run it but all instances of it
use a global section to communicate with each other.
You can't write it unless you have a privilege or all users
have the priviledge.
What happens if two people make private sections to the same
part of one file?
5) It should be more convenient to read or write one character.
6) The fact that you must allocate a fixed amount of quota
to a subprocess at creation time is a real screw.
It means you can't make two of them without screwing yourself
with a fixed partitioning. It would be better if they just
counted against you in an unlimited way.
7) No reasonable way to do tty passing (except with mailboxes).
8) Paging is done all wrong, of course. And there is no way to
ask about the state of your own address space.
9) Having system global sections have names not deduceable
from the filename screws people.
What happens if there are two system global sections that overlap?
10) It is a loss that event flags aren't just any bits in any words
in your memory. And, that they aren't sort of the same as io status blocks.
Also, that some things just use ASTs and events, while others
make mailboxes which you QIO up ASTs and events for.
11) Is it possible at all to use disk files not through RMS?
---------------------------------------------------------------------------
----Message 14 is----
Date: 6 May 1979
From: Miller at MIT-AI (Mark L. Miller, TI)
Subject: Thoughts on Op Sys for Vax
First let me explain that I am not yet a Vax user; ours is still in crates
out in the hall. Moreover, my knowledge of Unix is minimal. With those
caveats in mind, I meekly offer some suggestions...
(1) It sounds like neither VMS nor Unix is really suitable. Since so
little software currently exists for the Vax, perhaps it is really sensible
to consider a brand new Op Sys. Perhaps it could be based on a "multi-user
Lisp machine" philosophy. That is, implement a NIL virtual machine
(minimal Lisp subset) in micro-code, and then implement the entire thing on
top of that, including a decent tree-structured file-system and a
Lisp-syntax command interpreter. This is a serious (even if naive)
suggestion.
(2) I heard a rumor that "Tops-Vax" is in the works at DEC, tho I forget the
source of the rumor. If true, and depending on what that means, it warrants
consideration.
(3) Full support of real-time features is essential for widespread acceptance
of any new Vax Op Sys. Many labs requiring data-acquisition are getting Vaxs.
(4) Fully public domain software is essential. Our experience with Bell labs
has been that they are not anxious to see us obtain C or Unix.
(5) Substantial, accountable support is essential. Something which is
home-brew at just one site, and not readily subject to influence by other
sites (e.g., ITS on the 10) is hard to sell to outside sites. How many
people have had to live with Bottoms-10 because those in authority at their
10 sites did not trust MIT's decision making processes? (And there are
probably lots of useful user-written tools lurking in the bowels of various
Bottoms-10 systems.)
Thanks for listening. Please put/keep me on this mailing list. -- Mark
-----------------------------------------------------------------------------
----Message 15 is----
Date: 6 May 1979
From: GEOFF @ SRI-KL (Geoffrey S. Goodfellow)
Subject: Source Availability
I just preused your VAX.TXT file, and have one comment to make.
That is with respect to the VMS system sources, et al. I think
everyone just assumes that everyone who runs VMS will have the
sources for them. While this was the case with TOPS-10, it is
not the case with TOPS-20, for which DEC charges 40k$ for the
TOPS-20 and EXEC sources, another Nk$ for the Front-end sources,
and then a whopping 100K for the microcode sources, while UNIX
comes complete with sources when you get it. Hence, you can't
just assume that everyone who has a VAX is going to have the
sources for VMS. I myself do not know if DEC is going to charge
for the VMS sources, but I would suspect they are, since they
know that is where the big bucks are, in software.
----------------------------------------------------------------
----Message 16 is----
Date: 6 May 1979
From: WILCOX @ SUMEX-AIM (Clark R. Wilcox)
Subject: MAINSAIL VAX
This is just a note concerning a machine-independent high-level language,
MAINSAIL. We expect to undertake a VAX-VMS implementation starting in
late summer '79. In general, it is not too difficult to re-target for
another operating system, and we are basically "in the business" of
doing this; thus the VMS versus UNIX issue is not crucial to us.
MAINSAIL has been in development for the past 5 years or so, and is now
running (though not necessarily in general distribution) on TENEX,
TOPS-10, TOPS-20, RT-11, RSX-11 and UNIX. A number of other machines and
operating systems are waiting their turn. MAINSAIL has evolved from the
programming language SAIL, and while it has a very SAIL-like flavor
(e.g. strings), it is certainly not simply "SAIL made portable" (if such
a thing were possible). For example, MAINSAIL has a well-developed
notion of module, and these modules are position-independent and
dynamically linked (there is no link step as in SAIL). The fact that
MAINSAIL does not rely on host-system linkers makes possible the porting
of large programs consisting of hundreds of modules.
MAINSAIL is of particular interest to the DEC community since programs
can be made portable across 10's, 11's and VAX's. In particular, program
development for a not-yet-devlivered VAX can be started right away on,
say, a PDP-10. Since 99% of the MAINSAIL system is itself written in
MAINSAIL, the same software is run on all implementations, and hence the
incompatibilities caused by multiple implementations of the same
language (as in PASCAL) are absent.
The only remark I have concerning UNIX versus VMS is that the UNIX
operating system interface was trivial, whereas the RMS approach taken
by VAX (and which we have already dealt with on RSX-11) is a real pain.
This is because MAINSAIL views files as a sequence of cells (characters
in the case of text files, "storage units" in the case of data files),
and the line counts and such inserted by RMS simply get in the way.
I encourage anyone concerned with the choice of a programming language
on the VAX (independent of the operating system), or for that matter on
any "reasonable" machine (typical 8-bit micros are not reasonable) to
get in touch with me either by net mail (WILCOX@SUMEX-AIM), by phone
(415-497-3654, my office at Stanford), or by mail (TB-105, Stanford
University Medical Center, Stanford, Ca. 94305). We are especially
interested in programming projects oriented toward portable software.
-----------------------------------------------------------------------
----Message 17 is----
Date: 7 May 1979
From: RWK @ MIT-MC (Robert W. Kerns, MIT)
Subject: Notes on VAX/VMS
Here are several points, oriented toward VAX/VMS, which I've used more than
UNIX-11 and am more familiar with. I have not used VAX-UNIX, and my total
knowledge of it comes from Fateman and your file.
Note: Take care in reading this. I try to clear up many points made against
VAX/VMS with incomplete knowledge. This will sometimes sound like an attempt
on my part to defend the entire system. MANY OF THE POINTS MADE REMAIN VALID
despite the overlooked strong points. VMS is a mixture of winnage and lossage,
and I'm merely trying for balance with what has been said before, not for a
comprehensive review. I don't have time to re-affirm all the bad points
already made. Also, I think in some cases it has gotten bad press by UNIX
people who happen to like such-and-such a UNIX feature and didn't fully
understand the VAX/VMS feature, and so didn't like it. I know I've reacted
similarly on occasion, and later had to revise my opinion. (My knowledge of
UNIX also predates my exposure to VMS)
My knowledge of UNIX-11 is primarily editing with some knowledge of the
internals due to having been told them by a heavy UNIX-internals hacker, plus
having read the manual. Primarily things like the schedular and the disk
system. I have seen code for both UNIX-11 and VAX/VMS, but I'm far more
familiar with the later. On VAX/VMS I've been using it since the first of the
year, non-fatally (so far!). At first it was very hard to deal with. With
various aids we've written it's somewhat managable now, but it sort of a
stilted style unlike what we adapt on ITS, where we can acomplish what we want
instantly, or on TWENEX, where I can figure out how to do what I want without
to much trouble, and where what I find is natural and makes sense. I find
myself avoiding certain types of things on the VAX. However, my style of
operation on VMS is more like that I adopt on UNIX than on ITS or TWENEX.
1) I don't think you want to remove RMS from the system, or subvert it,
or anything like that. I think you want to extend it! I think you want
to extend it to understand normal (our conception of normal, NOT DEC's!)
files, and DEC's files, and interface between them. This means that a
normal file (identifiable by a bit in the file header) read by a DEC program
would see the file as a variable-length-record file, and your text editor
could see a byte-array. I think these could even co-exist with each other
fairly well, with only an occasional curse thrown DEC's way.
2) VMS is VERY reliable. Essentially all the crashes I've seen in months
of use have been caused by running buggy privileged programs that got errors
running in kernal mode. (By the way, the capability for a user program to
run in kernal mode, if so privileged (program or user) can be quite useful).
(Oh yes, also a few crashes because some acoustic foam blocked a vent, causing
the airflow sensors to shut down the machine...until I removed the foam.)
3) I wouldn't term the VAX filesystem fragile, but it does share some
flaws with the UNIX one, such as the shit-for-brains feature of links working
by entering the same file identically in different directories. (Or
differently in the same directory, etc.). An example of the screw: If one
person deletes the file from his directory, it is gone, and the other person(s)
sharing the file get an error on that line of their directory listing.
Presumably until someone creates a different file that uses that INODE (in UNIX
terms) or file header (in VAX terms). At least DEC doesn't advertise this, but
plans to implement symbolic links. (The hooks are there in the disk format
and it might possibly work, but we haven't gone to the trouble of doing the
low-level stuff to try it out.).
4) Extended access-control mechanisms may be forthcoming from DEC. The hooks
exist in the file format.
5) DCL (equivalent to the SHELL in UNIX) has *NO* search rules. It looks on
your connected directory for the program to run, unless you explicitly tell
it where to find it. The standard way of dealing with this lossage is to
define symbols which include the full pathname, for each program in the world
you would ever want to run. This can get so slow that RLB has written a
program to define all the symbols en masse, which speeds up your init file
considerably.
6) There is the concept of a connected directory, but no concept of a home
directory. Thus programs like TECO generally get their init files from your
connected directory, which is an incredible screw. We got around it by
patching TECO to get the init file by reading from a logical device. Home
directories COULD be done by programs having a convention of looking for
the init on HOME: ... this is managable, and maybe even the right way, but
DEC didn't think of it!
7) I don't think that VMS is too awfully obscure. DEC has reformed from
the practice of assembly language programming, and programmers are forbidden
its use unless there is SPECIAL REASON (or so I've been told by people in
DEC). So there's hope for the future. This also means, however, that there
is a strong chance of ending up with a dinosaur like SAIL or CMU's PPN's.
RLB and I spent a few minutes once we had familiarized ourselves with the
conventions in the sources, and quickly came up with a patch to a problem
in the TTY driver. The code was fairly well commented, and we had our
patch right the first try (although we blew installing it a couple times,
there's no software to patch a running system that we know of, so we wrote
our own little program which changed it's access mode to kernal and modified
the system.)
8) All system calls in VMS are callable directly from higher language, as
subroutines, without ANY interface code. I don't know where Joe Newcomer got
his incorrect impression! This is one of the things DEC did right. They all
return status codes (an integer) which are system-wide, and can contain
information about what happened, whether successful or not. (I.e. a
success-return can also say that the IO is in progress.) There is a system
call to look up the meaning (in ascii) of a given status return. The low bit
being on means success. There are standard arguments given to the calls,
which can be strings (and even their Fortran has strings) integers of datatype
generated by the systems, and more complex structures, but ones that FORTRAN
can handle, albeit clusily. Note that this is * NOT * due to obscurity or
barouquity on the part of the system, but to the limited expressive power of
FORTRAN. C, it turns out, strikes out even worse, because of its inability to
deal with things like strings in logical ways. How many times have YOU
written a loop to count characters in an ASCIZ string in C? VAX/VMS and all
the languages DEC supplies deal with strings as descriptors including an
address and a length count. (Actually, they deal with a lot more things than
strings, and descriptors are a standard datatype. If a language can deal
with descriptors, it is home free with regards to interfacing to VMS system
calls.) BLISS, of course, wins out handily, as would PL1, even COBOL!
9) A C compiler which runs under VAX/VMS has been written here at M.I.T. by
a person in Ward's group. We do use a certain amount of UNIX software on the
Vax such as P (but not as a filter, unfortunately there's no easy way to do
that, although a smart command interpreter could make it work in a manner
not too disimilar from how the shell does it on UNIX) and GREP.
10) I would not expect to ever see a BLISS/UNIX unless BELL undertakes to
write it. The sources for BLISS are simply NOT AVAILABLE. Maybe if you
offered DEC $500K and signed away your life, but I suspect it would take even
more than that!
11) What is this TOPS-10 type search-list? Not logical names, they only
allow a single name, with no searching, unlike TWENEX. The command
interpreter only looks at the connected directory (not even your home
directory (no such concept)).
12) Fortran produces good VAX native code. Excellent, in fact.
13) BLISS-32 produces even BETTER code. Clearly DEC has been putting in
effort in the wrong places.
14) Native-mode software is being produced, DEC just released long before the
software and borrowed RSX-11 stuff. I would claim that the VAX/UNIX software
is for the most part ALSO pdp-11 software (of higher order) and that it isn't
what you want to see in VAX software either.
15) Interrupt handling is better in VMS. This is true if all the
things I've heard about interrupt handling under UNIX is true. In particular,
if you interrupt out of a system call, as I understand it, you get an error
when you continue the system call? And the code has to check for that error
and retry?
16) The Debuggers in both are deficient. I'm not too sure of what exists
elsewhere on UNIX, but what BELL ships is a file debugger, as I recall. The
VAX one has to be linked in in order to be used, you can't just say "DEBUG" or
"DDT" or something and start debugging your program. Also, it resides in the
same address space, which I'd claim isn't the right way to do it, but with
such a large address space it isn't too serious. ↑Y'ing (like ↑C in
BOTTOMS-10) out of a program with the debugger linked in (even if not used)
results in bizzare behaviour. (non-fatal, just annoying...you end up back in
the debugger of the program you tried to quit out of after one command, and
have to type ↑Z (end-of-file) to continue your command.
11) Almost all useful commands run a program, which means that there is
almost nothing at all (including listing your directory) which you can do
without aborting your editor.
12) The file system has version numbers, which are very very useful. Various
UNIX editors try to pretend to have this by renaming the old file "FOO" to
"FOO.BACK". Close, but no cigar. However, it would be easy to add to UNIX.
13) The command reader has a passion for uppercasifying things, just as UNIX
has a compulsion to reject typein that happens to be in upper case. The
former is worse, I think. It activly interferes on occasion. For example,
with using GREP! There is NO WAY to get at the un-uppercasified stuff, since
it happens
at the same time that scanning for variable substitution happens. I.e. if you
managed to get the un-uppercasified string, you'd also not have the variables
expanded properly, and quite possibly the command was a single variable that
had nothing whatsoever in it that you wanted; you want the expansion.
14) TTY interrupts (↑C, ↑Y, mailbox) This is done imperfectly on the VAX. It
is also done imprefectly on UNIX. You can't trap ↑Y yourself, you have to
have enabled it at supervisor mode. The DCL and the debugger run at
supervisor mode, but there's no way for the USER program to do it, unless you
have change-mode-to-EXEC privilege. However, you can set a table of what
characters to wake up on and to echo, etc. The way you get asynchronous
character interrupts is to have pending asynchronous I/O on a mailbox
associated with your terminal. Hardly the most convenient of things. And of
course, convenience is very important in TTY handling. I think UNIX does it
worse, though, since it evolved as a line-at-a-time system. I haven't used a
virgin unhacked UNIX ever, so I couldn't even swear it normally has the
capability even yet. There isn't any way to get an interrupt when ANY
character happens, is there?
15) Wildcards aren't "special cased" on program-by-program basis. It is easy
to use, in a fully standard manner. I've always considered UNIX's idea to be a
bit of a mixture of mis-feature and feature, since it enforces a single
command syntax. Sometimes you don't WANT wild-card processing; and the
command interpreter is not the place for it. It is provided by RMS in a
fairly straight-forward manner.
16) Very little TRUE VAX software (as opposed to adapted PDP-11 software)
exists at all. This is changing. The situation is somewhat analagous to
a year after the KA-10 came out. There was some adapted PDP-6 software,
but take a look how long it was before TENEX came along!
17) The on-line documentation is generally adaquate for the user-level
commands, but is not full documentation. It is possible to find out how
to interact with stuff, and what the various options to commands are. Also,
it is structured, as opposed to being all the documentation on the command
always listed, when what you want is the documentation on what the /LIST
switch does. I don't know if the online-documentation for UNIX is this nice
to you; I doubt it.
18) I agree that overlapped asynchronous I/O is not needed for time-sharing
applications. Real-time control is quite another story, however. People who
do anything like that with it would be sorely disapointed by UNIX. VMS has
a chance of letting them mix real-time stuff with timesharing, if their
real-time stuff isn't too demanding. The scheduler (which, by the way, is
quite clear and readable, despite being in assembly language. And is short,
too!) implements priorities in such a manner to give real-time response to
some stuff with absolute priorities over a certain level, while things which
don't need extraordinary response compete for the rest. And if you wire down
pages or the working set etc, you can expect quite good response (it seems to
me from examining how it works; I haven't verified all this empirically).
19) VMS requires you to place limits on your balance set, leaving up to the
user some of what should properly be set dynamically by the system. I.e. you
can page occasionally when you're the only person on the machine running a 200
page program. This isn't too serious, however, particularly if you have access
to raise your working set.
20) I hope that RJF and friends are going to provide primitives for locking
down pages, for swapping out of the entireity of programs which have dropped
below a minimum threshold of # of pages in core, etc. If not, their paging
is going to be inadaquate for a number of things (including efficient running
under heavy load (i.e. MACSYMA) and real-time stuff (I have doubts about
real-time stuff under UNIX anyway, but that's not what I'm looking for from
UNIX)
21) If you're trying to avoid learning the assembly language, be warned that
the debugger is going to insist! It does know about line numbers in FORTRASH
and probably something similar under BLISS-32 (I'm going to go get manuals
for that today, but haven't written any programs with it yet).
22) UNIX software HAS been influenced by the smallness of the -11. So has
VMS software though. The influence is entirely negative, in that the
software sacrificed featurefulness for smallness. For example, an editor
with the FULL scope of EMACS is simply not feasible on the -11. So much of
the lack of winning software will NOT be cured by going to UNIX, all you'll
get is better -11 software, not winning VAX software.
23) The shared page as communication is indeed scrod, as RMS suggests. The
only fix is to give all users access to create global sections, and bitch at
DEC. Essentially, by making it unusable if you enforce the security on global
sections, they remove the security!
24) I'm told the IBM/BELL problem is real, and shouldn't be laughed at if you
get to know internals of UNIX and ever wish to work for IBM. On the other
hand, if you have a healthy desire to not work at IBM, it's not a problem.
25) The command interpreter: It IS possible to do powerful things in it. We
have several programs written entirely in DCL, such as a command to manipulate
directory defaults that includes an 8-level stack of directory defaults,
relative addressing (i.e. "go up a directory level" or "go down this
sub-directory". ..) and various other features, such as a filename parser, for
use by other command files to parse the filenames so they can do processing on
parts of the filenames. I haven't written any major shell command files, but
it seems to me that DCL has a slight advantage here for a lot of things, but
both are somewhat deficient compared to Multics, which is also less than ideal.
26) RMS isn't as hard to use as has been represented. It's primarily a
learning barrier. RLB and I can generally get it to do what we want pretty
easily. It's even not too hard to do byte-stream stuff. You just treat the
record as a buffer, and when you empty your buffer, you read in a new buffer
and give back a new-line character(s). It COULD be simpler, but not an awful
lot. And of course, RMS could easily be extended to leave straight ascii files
alone and convert record-files to byte-arrays.
27) It has been claimed that because UCB can get drivers up on UNIX and DEC
can't on VMS that therefore it's easier to bring up a driver on VAX/UNIX than
VAX/VMS. This would be true if the programmers and their environments were
comparable. They aren't.
28) I think DEC may plan to add on command completion ala TWENEX. If not,
they're stupider than I thought.
29) Increasing the number of subdirectories is harder than you think, but
probably feasible (Some applications software will have to be recompiled).
However, 8 deep isn't all that bad. It's deeper than I've ever seen anyone
use on UNIX! (By 3 levels!) (By the way, I actually have some circular
directory structures...as I'll mention later this doesn't bother anything at
all, but that's not a feature!)
30) Another filesystem problem I forgot to mention earlier. If you delete
a directory containing files, NOBODY EVER RECLAIMS THE SPACE! You have to do a
disk-to-disk copy (involving tape intermediary if you don't have two drives!)
to get it back! Only explicit deletions of a given file make it go away (even
if it's shared!). And of course, there's no way to find out how many times a
file is shared other than looking over ALL directories for things pointing to
it (and of course that doesn't catch timing errors). So you don't make links
or you put up with the hassle. An almost-collorary of this is that nobody
cares about the directory structure. It's there to FIND files, not to KEEP
them. So if you have loops in your directory structure, anything will punt
after following it 8 levels, not caring that you have circular structure.
Try THAT on UNIX! (And, of course, there's no UP link either, which is both
OK, since you just strip off one component of the directory name to get at
least one superior of the directory).
31) I'll be writing a network mailer which operates via the CHAOSnet, if
anyone is interested. Unfortunately, we don't have our CHAOSnet interface yet,
so I'm not actively writing it.
32) It would be very easy to replace individual VMS modules with BLISS-32
and interface cleanly with the rest of the system. There already IS code that
does this. For a lot of things you could even do it in Fortran, if you bring a
note from your psychiatrist.
33) I've heard about more UNIX disks needing patching in the last year
than -10's, and I know about more -10's. I suspect that UNIX could be brought
up to par, however. Most of the lossages I've heard about have been bashed
free-lists. Since on UNIX any block can be in the free-list, if a wrong one
is in the free-list the first several words can point to un-free blocks. One
bit of error leading to the wrong next block in the free-list can condem a
large number of blocks to the scrap heap. A bit-map in a fixed place isn't
quite as sensitive to such errors. However, it should be easy enough to
change to a bit-map. Maybe it's not even necessary, I don't know.
34) A command interpreter COULD do I/O redirection ala UNIX. Programs read
from SYS$INPUT and write to SYS$OUTPUT or SYS$ERROR. That's the mechanism the
shell uses, too, as I understand it.
35) "5. VMS drivers are difficult to add and a mess, as is the entire system.
We have added several drivers to UNIX easily. Looking at a VMS driver
is a waste of time."
I dunno, I can understand them well enough. Not as good as UNIX, true,
on the other hand, I think the above overstates the case.
36) Take all the things I DIDN'T say (and some of the ones I did) and that's
what is wrong with VAX/VMS. The above are the things which occurred to me
while reading the comments the people who haven't used it, as one who has.
I haven't edited this much, since I've spent all night almost composing it,
but I hope it's of some use. In short, I don't think that VMS is quite as bad
as the UNIX people paint it. It has SEVERE problems, none-the-less. The worst
of these I think are the multiple-process-controlled-by-a-single-command-
interpreter related problems. If DEC comes across on that issue, it will help
more than any single other issue. I don't think that UNIX has proven itself
capable of growing to the extent needed either, but I don't wish to discourage
advocates of either system from proving their point!
In short, I await release 2.0 of VMS to pronounce it dead (1.5 is out now,
I believe, but I haven't seen it yet.). And I await more detailed information
and future development of VAX/UNIX. And if someone has the resources to throw
them both away and start from scratch, or do a major overhaul of VMS's outer
layers doing it right, then that's the best course, IF HE HAS THE RESOURCES
TO BACK IT UP! I expect we'll put NIL on whatever turns up, if it's
adequately widespread.
----------------------------------------------------------------------------
----Message 18 is----
Date: 8 May 1979
From: CROSSLAND @ UTAH-20 (Jim Crossland)
Subject: Note on DEC-10/20 Future
I was recently pointed to the file VAX.TXT at CMUA and ask to read it.
After reading the first sentence it was apparent why I was asked to read it.
That sentence was "It appears that the future computing needs of the major
ARPANET laboratories will not be met by the PDP-10/PDP-20 architecture
because of its limited address space and decreasing commitment to this
architecture by DEC."
I am not trying to throw a wet blanket on your VAX development as
I am aware that you already have your system and from reading the rest of
the file you seemed to be doing the right thing, in my mind at least, of
trying to solve your problems with VAX. The purpose of this note is to
try to correct some bad information that was being given out by people
within DEC about the Decsystem 10/20 line. As you may or may not be aware
the situation has come to a full head in the last few weeks and appears to
have been resolved, in that Ken Olsen has written a letter informing
people that the 36 bit line is not dead, has not been canceled that
there will be a new machine in two years and that there will be continued
software development. In addition I have it from good sources that there
is still work being done on the entirely new processor. The process due
in two years is just a faster version of the KL, no real new features.
As for address space with release 4 of TOPS-20 the user will have access
to all 32 of the 256k sections. This has not been announced as part of
release 4 software since it is not complete (currently the PSI system
can only vector to section 0 where you may transfer to another section
if you desire). The user extended address facility is scheduled for
completion and formal announcement in release 5, but there is no reason
that software development cannot start in release 4. It has been stated
that this implementation of extended addressing will be awkward and hard
to use, this is not true. It is true that in order to maintain compatibility
section 0 is special, but once out of section 0 the extended address facility
is no harder to use that that of VAX. Most extended programs probably
will not use section 0 at all thus avoiding the problems of transfering in
and out of section 0.
I am aware that many people have been given incorrect information
about the fate of the TOPS-20 operating system at DEC. As you may or may
not be aware steps are being taken from within DEC to correct this
situation and see that it ceases to happen. As a former member of the
TOPS-20 Development group at DEC I feel that it is too bad that all of
this has happened, it has surely hurt the TOPS-20 and I feel will
eventually hurt the VAX system (due to loss of credibility of the people
within DEC promoting it). I too have heard the rumors of TOPS-VAX and have
reason to believe
they are true, but I do not know for sure. I think that this would
be a step in the right direction for DEC since my feeling is that many
of the people working on VAX at DEC lack large system experience and
the influx of people from the Large System Group would help correct
this problem. That is not to say that LCG people do everything right,
but that you would do things differently on a small or medium size
11 than you would on a larger system with much more memory and address
space. If VAX is to be successful DEC is going to have to stop treating
it as a mini and start treating it as a large computer.
I do not know if this information is useful to you, but thought I
would make a plea for the end of the spreading of incorrect rumors.
Thank you,
Jim Crossland
---------------------------------------------------------------------
----Message 19 is----
Date: 8 May 1979
From: David.Smith @ cmua
Subject: Notes on VAX/UNIX
Note: Most of the following points are in response to questions or
statements in preceding messages. I speak as a Unix user, not having
used VMS.
(1) Any more "Unix wedded to small address space" arguments should
refer to VAX Unix itself, not inherited user-mode programs. What
we are interested in is determining the limitations which will be
placed on us in the future. Anyway, the issue of the stunted
growth of the standard Unix editor must be balanced by the fact
that the VMS offering is SOS. We would need to get a real editor
in either case. (EX maybe?)
(2) Although the VAX is currently short on language systems, it
appears that before long will be available Ada, Mainsail, C, and
Bliss-32, regardless of which operating system we adopt. CMU
shouldn't have much problem getting the Bliss sources.
(3) There have been several negative comments (not all in this file)
about the Unix on-line manual. Some of these comments suggest to
me that the detractor doesn't use the CMU extensions to the
on-line manual system. The "key" command does keyword searches
through the documentation title lines, so you don't have to know
where things are to learn about them. The "man" command will let
you specify which section(s) you are interested in seeing.
(Section names are stereotyped.) Sections which are unlikely to
be frequently needed can be hidden, so that the bodies won't be
printed unless you request. But all section headings are printed,
so you know what you are missing. As for the slow speed of the
"man" command, consider this: IUS' raw speed is degraded by
one-third compared to a standard 11/40 by the delay of the
crosspoint switch. Additionally, the processor mods cause the
supervisor overhead to be approximately doubled. The VAX is
faster than an 11/70, and its overhead is slashed by 70% compared
to 11/70 Unix (sez Reiser). The larger size of the VAX might let
us find manual directories more frequently cached in main store.
So the "man" command should be a fair bit faster than what we
have been living with. (This is not to argue that the present
system can match ZOG.)
(4) Aspersions cast upon the VMS scheduler should be tempered by the
fact that the Unix scheduler (version 6, anyway) is at least as
dumb. They both use a fairly simple aging algorithm.
(5) Unix does allow per-keystroke process awakening. (How else do
the editors work?) It is called "raw mode". Unfortunately, you
can't set a break table: you get awakened either on each
character, or on each line (cr, lf, esc terminators). Another
loss is that when the program switches between raw and cooked
mode, the input buffer gets purged. This makes GDP hacking (for
example) miserable.
(6) I/O redirection: VMS accomplishes it at fork time by the
SYS$INPUT, SYS$OUTPUT, and SYS$ERROR logical name parameters.
With Unix, redirection occurs between fork and exec times, and
open file descriptors are passed. A file descriptor may be for a
pipe or a file, and the child program doesn't have to know which.
Since the parent can seek to the end of the passed file (or
anywhere within it), it can cause the child to append to the
file, and the child won't have to know about it. Pipes and
append mode are not transparent to VMS processes.
(7) Re: "there is almost nothing at all (including listing your
directory) which you can do without aborting your editor" [said
of VMS]: Couldn't Twenex-style push/pop context be achieved by
having the command interpreter fork another command interpreter
and wait for it to exit?
(8) The "ddt" debugger on our systems can redirect I/O and trace
program execution. It is uniformly better than "cdb" (although
nothing short of source-language debugging is fully
satisfactory).
(9) The Unix pipe mechanism should be modified to avoid flushing to
disk. The VAX should be big enough to keep pipes in main memory.
(10) Each Unix program comes equipped with a SETUID bit, which (if
set) causes the program to be executed within the protection
context of its owner. The program can still tell who invoked it.
We have some data bases (e.g., Cheese and Csd) which should be
modifiable by anyone, but ONLY through a special program (NOT the
editor). Setuid makes such protection trivial under Unix. How
do you get this functionality under VMS?
-----------------------------------------------------------------------
----Message 20 is----
Date: 8 May 1979 1613-EDT
From: BRUCE.LEVERETT at CMU-10A
Subject: VMS vs. UNIX -- the promised flame
To: SCOTT.FAHLMAN
Origin: BRUCE.LEVERETT(C410BL03) at CMU-10A; 8 May 1979 1613-EDT
Considerations in file formats
This is not a contribution of new facts about VMS or UNIX, but more in the
style of a flame. The issue of file formats has been raised in previous
contributions. I would like to introduce a perspective about what is good
and what is bad in this area, that many readers may not be familiar with.
Several contributors evidently prefer the file format offered by UNIX to
the various formats offered by RMS. Gosling, Shafer, and McKeown speak of
the "clean and elegant" UNIX byte-stream format and ask, "How often does
one need fancy file formats?" The file format of Unix is described in
Kernighan & Mashey's article, "The UNIX Programming Environment" (Software
Practice & Experience, Jan. 1979):
As far as the file system is concerned, a file has no internal
structure -- it is a featureless, contiguous array of bytes.
Like Gosling et al., Kernighan and Mashey believe that this is a characteristic
of the system whch is critical to its success.
The point of this flame is to suggest that this view of file formats is
bogus, that is, that endorsement of it costs us time and money every day.
We don't notice the costs because we don't know what we're missing. We
thnk that we don't need the file formatting offered by RMS because we don't
use formatted files, or because we don't run data base applications. But
we are like the protagonist of "Le Bourgeois Gentilhomme", who learned with
amazement that the stuff he had been speaking all his life was Prose. We
use many of the same abstractions that RMS users use, but we aren't aware
of it. And since we aren't aware of it, we must constantly reimplement the
same abstraction tools, at great cost to ourselves. Let me give three
examples.
(1) The most important and often-used UNIX programs do not use the basic
file format, but implement an abstraction on top of it. Files are
considered to be sequences of "lines". Programs must do four things
to implement this abstraction:
(a) They must watch for the CR-LF sequence; it is the designated marker
for end-of-line.
(b) They must output the CR-LF sequence at the end of each line.
(c) They must do something about naked CR's and about naked LF's.
(d) They must deal with files whose last characters are not CR-LF.
All programs, including "grep" and "awk" and "diff" and every single user
program that deals with text files, must reimplement this abstraction.
This is also true, of course, on all TOPS-10 programs, since TOPS-10
also offers byte streams. Many programs, of course, do not handle (c)
and/or (d), leading to anomalous behavior.
This abstraction is probably the most basic one offered by RMS. You need
not search for the end of a line; RMS will tell you where it is.
(2) Many readers of this file have heard of SOS, a line-oriented text editor
on many TOPS-10 systems. This editor has lost a lot of its appeal since
the advent of CRT text editors, and rightly so. I will use it as an
illustration, however, in hopes that people will understand clearly the
point I am trying to make.
SOS offers an even neater abstraction than lines: line-numbered lines.
In DP jargon, these are called "keyed records". These have various uses;
for instance, when a compiler discovers an error in the file and calls
SOS to let the user fix it, it can set the user down at exactly the line
where it found the error.
Unfortunately TOPS-10 offers only "byte streams" and line-numbered files
must be implemented in terms of "byte streams". This has led to
unimaginable headaches of implementation. Every compiler on the PDP-10
must be able to deal with these weird files containing line numbers
(implemented using the extra bit left over when 5 7-bit bytes are packed
into a 36-bit word), funny page marks, and lots of null characters. SOS
itself must implement key-search as linear search through a file, and
that's only the beginning of the hackery inside that editor. Naturally
many system programs do not understand SOS file format, rendering both
them and SOS less useful. Many other programs have bugs in their
understanding of SOS file format, leading to unexpected lossages at random
times by random users.
Here again, RMS has something obvious to offer. Perhaps SOS would not be
so unpopular, all over the ARPANet, if ARPANet sites had file systems which
provided support for it.
(3) CMU users, and perhaps others, are familiar with the RDMAIL mail-reading
system. (Other ARPANet sites have their own, such as HERMES and MM.) This
is a standard data-base manipulation system, of a kind that is ordinarily
written by B.S.-level random programmers in the DP world. At CMU, however,
it was written by a Phil Karlton-level Phil Karlton, and is maintained by
a team of ultra-wizards comparable to, if not equal to, Phil. Why is this?
The problem is, once again, byte streams. RDMAIL works on very highly
structured message files. Its primitives include message Insertion,
Deletion, Classification, and Sorting. If the file formats offered by RMS
could have been used, each of these would map neatly onto the available
operating system primitives. But TOPS-10 offers only "byte streams".
Phil had to design his own file format, and it has been redesigned in
several major and minor ways since the beginning. The primitives were
written in hard and messy SAIL; instead of having a 10K high segment,
RDMAIL has a 43K high segment.
Let me recap what I have said with these three examples. I think that file
formatting is a useful data abstraction. It is so useful that we have
implemented it ourselves, in spite of the greatest difficulties, over and over
at every site on the ARPANet. I think that in terms of what we need from a
file system, we are not so different from DP programmers -- we are simply
less aware of our own needs, because we have always used our limitless
hacker-power to satisfy them. The proper way to face the future is to
acknowledge that, since 50% of our cycles are spent in doing file formatting,
we might as well have it done in the guts of the file system.
----Message 21 is----
Date: 8 MAY 1979 1952-EDT
From: JNC at MIT-AI (J. Noel Chiappa)
Subject: UNIX v VAX
To: SEF at MIT-AI
Via: MIT-AI; 8 May 1979 1951-EDT
I believe I am the 'UNIX system hacker' referred to in
RWK's note, and I will here rise to adjust a few points about
UNIX. I am not being evenhanded in this at all, merely commenting
on few things he said.
1) Deleting one link to a file in UNIX does not leave all the
other pointers pointing to thin air (or worse) garbage. This may
happen in certain (rare) disk lossages, but I have never seen one
that was not caused by my own stupidity and hamhandedness (this issue
UNIX disk reliability keeps popping up!! Its a NON ISSUE!!!
UNIX disks often come to rest in a nonconsistent state, but there
are many programs that non system hackers can use which are
easy to use to fix things, and anyway it doesn't happen often.
Grumble... Foo. You ITS hackers have no right to be complaining..
I was fixing UNIX disks after I had been on the system for 1 month,
while I wouldn't go near an ITS disk and I've been using it for 2 years.
People who live in glass houses, etc...)
2) Yes, all the VMS system calls are directly callable from a higher
level language, but the call blocks are so incredibly baroque....
C does provide some support for strings, and there is a universal
representation for them, which does not stop you from inventing
your own. UNIX calls also retunr system wide error codes, and
there is a routine in a library for printing them, just like VMS
an MULTICS, to name two....
3) The ADB debugger (which may be V7 only, but I would guess that
VAX/UNIX has it) is a full capability debugger which will work
on files, core dumps and running processes. You can install
breakpoints, single step, send interrupts, etc. It is not
totally winning, nor will it do all the things that ITS and
DDT combine to do, but it will do many that they don't. You
still sort of have to know machine code to use it, so it's not
as winning as (say) MULTICS debug, which lets you debug in PL/1.
I find it adequate, and no special preparation (other than
leaving the sysbol table in if you want to reference symbolic
locations) is needed.
4) Interrupts on UNIX are less than totally winning, I will admit.
An interrupted system call returns a characteristic error status,
not whatever it was that you said Bob. Notice that only some calls
(e.g. sleep, wait for a subprocess, read from a tty) are interruptable.
The others ruin to completion. The whole problem of interrupts inside
the system is a real pain in almost every system. I guess in MULTICS
you can't blovk in ring 0, so all interrupts take place in ring 4
(don't quote me on this...) which is one way of ignoring the
problem. Multics also has VERY classy interrupt handling stuff
(I know one person who is running n different sets of software
out of ONE Multics process, using something arcane called event
call channels) ITS does backout as I recall, how well and in what cases
I don't remember.
You are free to handle evry kind of interrupt in UNIX, except
one special 'non catchable' one. You can do char at a time input
with interrupts on, although you have to beat on your tty code.
5) Wild cards are a perennial bone of contention. The shell wildcarding
CAN be turned off (quote your arguments). I happen to like not having
to do it myself. There are a few screws (the perennial favourite is
"rm * bak") with the shell doing it, but on the whole I like it.
6) Can you really pipe I/O from one command to another the way the shell
does it? I guess you can kluge aroung this with intermediate files,
so it's not totally losing. The VMS people made no effort to utilize
the capability, though.
7) I doubt if you've ever seen some of the REALLY hairy shell command files.
I bet it's at least as good as VMS. Take a look at a John Pershing
command file sometime. On our UNIX, the reporting half of
PFTHMG and the GFR are command files!! It is about an order of magnitude
more complicated than ANYTHING that I have any use for.
8) Two MIT hackers, I and Jeff Mogul, wrote drivers for identical
hardware, I for UNIX, he for VAX. They DID NOT take the same amount
of time to write. So much for that point (see my earlier note).
This should hold you UNIX haters for a while. Send me more
complaints - I'm happy to flame. My apologies for any misrepresentations
of VMS, ITS, or your favourite TS systemthat have crept in - if you
find any, please castigate me ASAP.
Noel
----Message 22 is----
Date: 10 MAY 1979 0400-EDT
From: WNJ at MIT-AI (William N. Joy)
To: fahlman at CMU-10A
Via: MIT-AI; 10 May 1979 1003-EDT
More stuff, separate from my editor remarks to rms:
1. In version 7 UNIX there is no lossage when switching tty modes.
There is actually a third mode, the three are:
cooked line at a time input, interrupts, etc
rare (cbreak) character at a time input, but with interrupts still
raw character at a time input, no interrupts, 8 bit paths
The screen editor (VI) runs in cbreak. If you used one process as an input
process (to gather up characters) you could run in raw, and have asynchronous
editor processing going on and stoppable as soon as something was typed.
This is fairly cheap in UNIX (the `edtv'' program which Reiser wrote which
is a UNIX version of th SAIL editor uses this strategy with much success).
When you switch tty modes you get to choose whether to throw away typeahead
and wait till output flushes. This is messy in transitions out of
raw but works find on rare->cooked and cooked->rare transitions.
(This tty driver isn't half-baked, its ``fully baked'').
2. There IS a source language SYMBOLIC DEBUGGER on VAX. It is called SDB
and does not require any knowledge of ASSEMBLER (machine language if you will
since UNIX considers use of machine language in poor taste:
``Assembly language: This language is dead. Necromancy will
be severly punished.''
--- Introduction toUNIX/32V manual
In any case, SDB is symbolic, so that when you hit a breakpoint you
get a source level backtrace of calling routines, can look at variables,
invoke procedures (not interpretive, but actually runs the thing
in the child process, etc.) This is yet a modest debugger, but
with everything written in a clean and high-level language seems more than
adequate. You can also use ADB if you want to poke around in the machine code
stuff.
We are putting up a pascal translator with the portable C compiler and
see no reason why we can't make SDB work both with Pascal and C and Fortran77
all at the same time in any mix. The design is simple enought and elegant
enough to allow this. (sdb is actually SMALLER! than adb...just removing
the instruction decoder (binary->printable format bought
back more space than the symbolic stuff took).
--- some misconceptions ---
3. There are no CR's in UNIX'es files. Only newlines (12's in octal).
The system supplies what the terminal needs when outputting the file.
4. The "Standard i/o library" used byy all new C programs and utilities
provides primitives to read through a file a line at a time.
It is true that behaviour when a trailing new line is missing is erratic
if a program expects complete input lines. This rarely happens
and nice programs will supply the missing newline rather than throwing
the partial last line away.
Thats it for now.
Ciao
Bill Joy
----Message 23 is----
Date: 10 May 1979 11:24-EDT
From: Robert W. Kerns <RWK at MIT-MC>
Sender: ←←←047 at MIT-MC
Subject: VAX.TXT and the -20 series
To: crossland at UTAH-20
cc: SEF at MIT-MC, RMS at MIT-MC, RWK at MIT-MC
Via: MIT-AI; 10 May 1979 1128-EDT
Your remarks regarding DEC's support for -20 seemed a bit of a non sequitor
to me. I think it's a waste of breath trying to convince people doing research
to buy a -20 now. It would be a disaster if they did. I trust your intent
was to reassure people who already had them that they weren't going to be
left in the lurch support-wise?
I don't agree that any conceivable -10 archetecture with extended address
space (or more accurately, multiple address spaces) can be easier to use than
the VAX, where it's designed in from the start. So I do think it's fair
to say that the PDP-10/20 archetecture CAN NOT fill the needs of the
research community in the future. Even not counting ease of use, it's
STILL not enough address space! The extended-address feature is not at all
suitable to a LISP environment, for example, where you want a SINGLE address
space.
However, that's not too important, since the VAX is close enough to an
adaquate support (except, currently at least, for the software!) that we'll
still be buying DEC products. I fully agree with your comments about getting
people from large systems involved with the VAX. VMS shows countless signs
of out-and-out ignorance of big system ideas.
----Message 24 is----
Date: 10 May 1979
From: TJT@MIT-ML (Thomas J. Teixeira)
Subject: UNIX vs VMS
1) The question of whether an operating system requires assembly language
write-arounds for interfacing system calls to high-level languages is
ridiculous! In fact, VMS uses write-arounds, but they just don't tell
you about it. If the system calls are to operate in a more protected
environment, they must invoke some trap instruction. Unless the high-level
language is augmented to know about system calls as another data type, then
assembly language write-arounds are necessary. Admittedly, the Version 6
PDP-11/UNIX write-arounds are not very elegant, but the reason for this
is historical; UNIX was around before C, and the interface to the system
calls was originally designed for assembly language, not for C. I would
be very much surprised if the VAX/UNIX write-arounds were not very much
simpler.
2) Running C programs under VMS is possible, but this doesn't mean you can run
arbitrary UNIX C programs under VMS without a certain amount of hassle.
We have a VAX C compiler running here (however, it is a modified Version 6
UNIX C compiler, and hence can only be distributed to UNIX licensees).
Simple programs can be ported alright, provided they had been written to use
the right I/O library on UNIX. However, many UNIX system calls are hard to
emulate on VMS, notably fork, exec and pipes. The VMS "equivalents" for
these are either non-existent, or at too low a level (e.g. mailboxes).
While it may not be too hard to either implement these as subroutines, or
add them to the VMS kernel, one should be wary about making any changes to
VMS that are not highly modular (e.g. changing the command interpreter, or
adding a device driver for pipes), unless we can get DEC to support the
changes. Otherwise, we will either be constantly adding the same changes to
new releases of VMS, or (even worse), ignoring the new releases of VMS in
order to avoid putting in these changes.
3) One problem with VMS that has not been mentioned is the baroque privilege
mechanism. In many instances, the privilege required to perform relatively
innocuous operations is excessively high. For example, you need PHYSIO
in order disable echoing on your TTY.
4) VMS does not dynamically allocate stack space to user programs. There is
also no way to do this in user mode, since VMS passes the exception to the
user as a signal, which gets pushed on the user stack, which gets a memory
fault, which causes a fatal error the second time around.
5) It is true that one way of looking at RMS is that it provides typed files,
as opposed to untyped files. This would be acceptable if RMS also provided
1) a byte stream file type and 2) coercions between file types. Without
coercions you get idiocies like files written by the command interpreter
be unreadable by the the command interpreter.
6) re Bruce Leverett's comments on record oriented file formats: It is true
that for some applications, viewing a file as a stream of lines is more
convenient than as a stream of bytes. However, at the lowest level, the
file is still implemented as a byte stream. RMS still has to read each
block of the file in order to find a random variable length record (although
it doesn't have to read each byte). The operating system could perhaps do
this slightly faster than an user program; if (as in UNIX) blocks are read
into the kernel and copied to user space, then many blocks would not need
to be copied at all. However, a user mode subroutine enjoys no such
advantage. In any case, TANSTAAFL, and using RMS does not necessarily
make seeking to a random line more efficient.
7) In summary, RMS may be fine for some applications (e.g. data base processing
with record locking, etc), but it gets in the way for text files.
----Message 25 is----
Date: 10 May 1979 12:52-EDT
From: Robert W. Kerns <RWK at MIT-MC>
Subject: Misunderstanding, etc.
JNC slightly misunderstood a few things I said in his note. First, he's
not the UNIX hacker I was refering to. Second, I was talking about VMS,
not UNIX, wrt what happens when you delete a file someone else it pointing
to. My point was that UNIX does it wrong, and VMS does this even WORSE!
UNIX merely leaves it in his directory, still owned by the original owner,
who might not even know it was still in existence unless he explicitly
looked to see. (On VMS, he can't even tell, since there aren't any
reference counts!) (I just looked at my original comment and see how JNC
misunderstood my thrust on this...)
I'm willing to accept your claim that the filesystem fragility
issue is a non-issue. (If nothing else, recovery could be made as automatic
as it is on ITS)
I said what you said about interrupts; system calls interrupted out of
return an error. As for what happens on ITS: Either the system call is
so quick to execute that it just finishes up, or it backs up, or it keeps
a progress count in the user's address space (which the user can modify,
if he wants to throw away output, for example). No exceptions, and the
system calls just look like instructions. (Even the VAX hardware doesn't
do this perfectly, for example floating-point exceptions can lead to your
not knowing the PC of the instruction that caused the trap. So you can't
even continue.)
I wouldn't call ADB a full-capability debugger. I AM willing to stipulate
that it's better than DEBUG (although you haven't shown me how, I consider
DEBUG to be a pain, so it MUST be better...). (BTW, I know the person
who's going to be working on DEBUG this summer at DEC; I intend to educate
him to what my ideas of what a debugger should be, feel free to dump pet
beefs about DEBUG (or debuggers in general) my way...)
----Message 26 is----
Date: 10 May 1979 1313-EDT
From: Scott.Fahlman at CMU-10A (C380SF50)
Subject: VAX1.TXT
The VAX.TXT file (and its MIT-AI clone, VAX > ) have now grown to rather
unmanageable proportions. This will be the last entry in VAX.TXT.
Future entries will appear in VAX1.TXT on [c380sf50] at CMUA and in
SEF;VAX1 > at MIT-AI.